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Abstract: We present a preliminary data-based assessment of measurement reliability of the 
commercial 14-electrode Emotiv EPOC™ EEG wireless system in distinguishing between 
electrophysiological states of emotional function, as compared to a standard research-lab 
stationary 32-electrode EEG system for personalized single-individual use. Individual observers 
completed two tasks designed to elicit neural changes in emotional arousal and valence while 
simultaneously recording their EEGs with both systems in separate sessions. Participants observed 
emotion-laden words from the ANEW database and images from the IAPS database, both widely 
used and validated databases for emotional processes in multidisciplinary research. The pattern of 
results distinguished between high and low arousal and valence states using the stationary 
traditional system, but not the commercial device. Also, the latter device recorded EEG band 
frequencies at a much lower resolution and frequency range than the standard system. These 
findings suggest poor validity when using the commercial device and therefore should be 
cautioned against in a research setting. 
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1. INTRODUCTION 

The identification and classification of human emotions 
using electroencephalography (EEG) is a major 
research focus for developing Brain-Computer 
Interface (BCI) devices. Translating 

electrophysiological signals from a person's scalp into 
quantifiable data (event-related potentials, or ERPs) is 
useful for programming computer systems to 
recognize emotional content in facial expressions and 
vocal inflections [1] and allows software-based 
intelligence systems to adapt their function to a user's 
overall emotional state in real-time [2], The need for 
cost-effective EEG systems that can reliably distinguish 
between emotional states is growing in importance; 
EEG systems with a greater number of electrodes 
enable researchers to provide higher resolution 
recordings of electrophysiological activity but also 
require a lengthy preparation time to arrange the 
electrodes and test the signal amplification. Public and 
private businesses seeking to employ BCI devices to 
enhance their operatives' functionality are often 
interested in EEG systems with fewer electrodes- 
minimizing preparation time while potentially 
sacrificing the quality and resolution of the 
electrophysiological signal recordings. Recent reports 
have demonstrated that there is indeed some research 
utility for consumer-grade devices such as the Epoc as 
compared to other similar devices such as the 
Neurosky MindWave [3], However both of these 
devices' performance is questionable compared to 
proper research-grade EEG [4], 

The push to incorporate classification algorithms 
within these devices is prominently associated with the 
alluring promise of personalization, single- 
individualized use according to which the BCI system 
might be calibrated and fine-tuned to the individual 
characteristics of the person's brain/mind (which 
assumes an "N = 1" approach). Our current 

investigation provides novel evidence against relying 
on consumer devices for research grade tasks, 
specifically in this case for measuring 
neuropsychological aspects of emotional states 
(stimulus valence and arousal). Most current research 


with consumer-grade devices is limited since it uses 
only the consumer device itself; failing to demonstrate 
validity and reproducibility of results by comparison 
with a juxtaposed research grade device. We 
overcome this limitation by directly comparing the 
data acquired from a consumer grade EEG to that of a 
research grade EEG under the same conditions using a 
within-subject design. 

We acquired two adjacent EEG systems for their 
potential application as a part of a brain-computer 
interface: the Brain Vision Easy Cap EC40 headset and 
the actiCHamp active channel amplifier system 
(henceforth Brain Vision of BV) as well as the Emotiv 
EPOC™ EEG wireless system (henceforth abbreviated 
simply as Epoc). The Brain Vision Easy Cap is a standard 
research grade 32-electrode headset, whereas the 
Epoc headset is a consumer grade device with only 14 
electrodes, but boasts a much more elegant user- 
friendly design. In addition, it transmits 
electrophysiological data wirelessly via a Bluetooth™ 
transmitter, bypassing the signal processing 
traditionally performed by an external amplifier. While 
preparation time is minimal (10-15 min on average, as 
compared to 45-60 minutes for BV), the Epoc headset 
does not have any electrodes placed along the midline 
of the head, and it uses two reference electrodes on 
both sides of the head corresponding with the parietal 
lobe (P3 and P4) as opposed to using a single ground 
electrode along the midline as reference in traditional 
EEG systems. Furthermore, the electrodes are placed 
on the scalp without any gel substance serving as an 
intermediate for electrical conduction, thereby further 
limiting the ability to perform EEG source analysis 
(approximating the anatomical source of temporal 
electrical activity). For this reason, a viable test of 
comparative measurement reliability with other 
systems can be done through measurement of global 
or whole electrical brain activity. This is the approach 
we took in this preliminary study. 

In this study, we evaluated the Brain Vision Easy Cap 
EC40 and the Epoc EEG systems on their efficacy and 
reliability in differentiating between varying levels of 
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two distinct dimensions of emotion, emotional arousal 
and valence [4], We presented human subjects with 
two experimental tasks designed to examine if the 
Brain Vision system could be used to distinguish 
between states of high and low arousal and high and 
low valence, based on patterns of event-related 
frequency band potentials (ERBPs) in response to the 
presentation of emotionally-salient words or visual 
stimuli. Importantly, we report novel findings regarding 
the comparable signal quality of each device and the 
inability of Epoc to reproduce the results seen using 
the BV device. 

2. MATERIALS AND METHODS 

2.1 Participants 

Participants were three healthy graduate and 
undergraduate students (mean age 24.66 years old) 
serving as trained research assistants. This study 
followed all ethical guidelines on human participant set 
out by the host institution. All participants were right- 
handed males with corrected-to-normal (N=2) or 
normal (N=l) vision, and none reported any history of 
neurological impairment or were currently using 
psychoactive medications. The average number of 
hours of sleep on the previous night was 7.5 hours. 
Testing was conducted from 10 a.m. to 4 p.m. with a 
one-hour break in between on two separate days. The 
pre-experimental short adult version of the mood and 
feelings questionnaire [5] were administered by an 
independent research assistant unaware of the 
hypotheses and goals of the study to screen mood 
differences or emotional changes before the 
experimental sessions between days. No remarkable 
differences were reported as all participants scored 
similarly (overall score range: 3-5) in the two days and 
consistently well below the recommended clinical cut¬ 
off (possible maximum = 26; clinical cut-off = 11). 

2.2 Experimental design 

The participants completed two tasks sitting in front of 
a computer monitor while undergoing ERP analysis 
using the 32-electrode Brain Vision EEG system, and 
repeating both tasks two days later while using the 14- 
electrode Epoc EEG system. In the first task, subjects 


observed a series of selected images from the 
International Affective Picture System (IAPS). IAPS is a 
large standardized database of images, where each 
image carries a rating based on its three dimensions of 
emotion: valence, arousal, and control/dominance [6]. 
Emotion-laden pictures that had previously been 
categorized by the target parameters of valence and 
arousal were used to designate the image as high or 
low arousal/valence, previously shown to correlate 
with known EEG signatures [7,8]. Similar to this 
research, a recent fMRI study has shown that the 
emotions evoked by these images effectively separate 
the emotions into distinct arousal/valence quadrants 
[9], The calibration for these emotion-laden 
localizations occurred across participants. We pre¬ 
selected 100 high arousal and 100 low arousal images 
(high and low arousal determined by their rating in the 
database) as well as 100 high valence and 100 low 
valence images for use in our study. We presented 
each participant with a randomized selection of 100 
images (25 high arousal, 25 low arousal, 25 high 
valence, 25 low valence) on a computer monitor while 
recording their electrophysiological activity via the 
Brain Vision headset. The stimulus presentation 
program began with a central cross fixation to help 
participants focus their gaze and reduce eye 
movements. After 200 ms, the first image was 
presented for 4500 ms, followed by a 1300 ms 
presentation of the cross fix preceding the second 
image. Participants were specifically instructed to try 
to form an emotional connection with each 
image/word presented and to remain as still as 
possible while viewing images/words. Images filled a 
48.3 cm (19") monitor with a 1024 x 768-pixel 
resolution. Letters were presented at a font size of 48 
points and presented in the center of the monitor. 

In the second task, subjects observed a series of 
selected words from the Affective Norms for English 
Words (ANEW) database. ANEW is another 
standardized inventory of normative emotional ratings 
for a large number of words in the English language; it 
was developed to complement IAPS [ 101 . Previous 
research using the ANEW has plotted the normative 
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ratings in 2-dimensional space and found the 
distribution effectively complements the IAPS using 
EEG [9], We presented each participant with a 
selection of words from the ANEW database on a 
computer monitor while recording their 
electrophysiological activity (the same procedure as 
with the IAPS images, this time with words); one series 
of words with the Brain Vision headset, and another 
series with the Epoc headset. 

2.3 Event-related band potentials and signal 
amplification 

EEGs were recorded for each participant during the 
tasks. The Brain Vision EEG recordings were obtained 
from electrodes held in place by means of an elastic 
cap (Easy Cap, Brain Vision, Morrisville, NC, USA) 
adhering to the ten-twenty system of measurements 
[ 111 with 32 active low-noise pin-type Ag-AgCI 
electrodes (1 mm each) amplified (sampled up to 1000 
Hz) and recorded reference-free. Standard external 
landmarks (i.e. the inion, the nasion, and left and right 
pre-auricular points) were the basis for the montage 
distribution with electrodes positioned as percentage 
distances from these landmark points (see Figure SI in 
Online Supplementary Materials ). All impedances 
were kept under 25 kOhms so that they provided an 
effective electrical signal with minimal noise. The Epoc 
recordings were collected as is with no additional 
materials or measurements other than the device 
itself. 

EEG signals were amplified, and low pass filtered at 
100 kHz via the actiCHamp active channel amplifier 
with a sampling rate of 1000 Hz. All electrodes were 
offline re-referenced to a common average reference 
(i.e. the average was subtracted from each electrode 
for each time point). Offline averaging was performed 
such that event-related frequency band potentials 
(ERBP ) were averaged separately for each stimulus 
type and condition for each electrode with an Epoch of 
-200 ms pre-stimulus to 1000 ms post-stimulus. ERBPs 
included the individual trials to code high and low 
arousal, high and low valence. For details and analysis 
outputs, please see Online Supplementary Material . 


2.4 Statistical analysis 

For our statistical analysis, it was necessary to evaluate 
whole brain ERBP because the precision of the 
electrode placement in the Epoc could not be 
confirmed. Hence, there is no validity without this and 
comparing electrodes of one system with the other is 
fruitless. Additionally, we used a Vincentization 
technique for the EEG data which involves the 
averaging of three or more subjects' quantile functions 
in order to obtain group quantiles [12 - 141 . This 
statistical technique allowed us to construct F values to 
evaluate differences between the five frequency bands 
adopting a by-item approach, where the electrodes are 
the unit of analysis (cases). 

2.5 Ethics statement 

Upon review of the submission materials, following 
Article 2.1 the Tri-Council Policy Statement Carleton 
University Research Ethics Board determined that this 
study did not require prior review or approval by the 
Board. Documentation and signed consent were 
waived. Documentation of consent was obtained by 
recording computerized log-ins during voluntary 
participation. 

3. RESULTS 

3.1 Distinguishing electrophysiological states of 
arousal and valence with the IAPS Task 

The two panels below (Figure 1) summarize the 
frequency band analysis for examining arousal and 
valence using the IAPS task. Figure 1A displays the 
distinguishing of high and low arousal states using the 
Brain Vision system and Epoc system. A two-way 
analysis of variance revealed that the observed 
differences in the arousal condition (high arousal vs. 
low arousal) was not statistically significant for either 
system; differences in frequency band (Delta, Theta, 
Alpha, Beta, Gamma) were significant for the Brain 
Vision system only (F (1,31) = 63.224, p < 0.01); and the 
interaction between arousal condition and frequency 
was also significant for Brain Vision only (F (1,31) = 
12.303, p < 0.01). Figure IB displays the comparison 
between high and low valence states using the Brain 
Vision system and the distinguishing of high and low 
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valence states using the Epoc system. A two-way 
analysis of variance revealed that the observed 
differences in the valence condition (high valence vs. 
low valence) was statistically significant for the Brain 
Vision system only (F (1,31) =19.157, p < 0.01,); 


differences in frequency band (Delta, Theta, Alpha, 
Beta, Gamma) were significant for Brain Vision only (F 
(1,31) = 72.550, p < 0.01); and the interaction between 
valence condition and frequency was also significant 
for Brain Vision (F (1,31) = 51.138, p < 0.01). 
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Figure 1. Distinguishing high and low arousal (Panel A) and valence (Panel B) between Epoc versus Brain Vision using event- 
related frequency band potentials (ERBP) with the IAPS task. Y-axis values are in squared microvolts (EEG Power). From left 
to right, EEG bands in x-axis correspond to standard nomenclature: delta, theta, alpha, beta and gamma. 
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Figure 2. Distinguishing high and low arousal (Panel A) and valence (Panel B) between Epoc versus Brain Vision using event- 
related frequency band potentials (ERBP) with the ANEW task. Y-axis values are in squared microvolts (EEG Power). From left 
to right, EEG bands in x-axis correspond to standard nomenclature: delta, theta, alpha, beta and gamma. 
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3.2 Distinguishing electrophysiological states of 
arousal and valence with the ANEW task 

The two panels below (Figure 2) summarize the 
frequency band analysis for examining arousal and 
valence using the ANEW task. We plotted the means 
of the frequency band amplitude, measured in 
microvolts (pV) averaged across participants and 
electrodes. Figure 2A displays the comparison 
between high and low arousal states using the Brain 
Vision system and the Epoc system. A two-way analysis 
of variance revealed that the observed differences in 
the arousal condition (high arousal vs. low arousal) was 
not statistically significant for the Brain Vision system 
(F (1,31) = 0.085; p = 0.772) nor the Epoc system (F 
(1,13) = 7.185, p = 0.019); differences in frequency 
band (Delta, Theta, Alpha, Beta, Gamma) were 
significant for the Brain Vision system only (F (1,31) = 
34.156, p < 0.01); and the interaction between arousal 
condition and frequency was significant for the Brain 
Vision system (F (1,31) = 53.339, p < 0.01) but not the 
Epoc system (F (1,13) = 2.191, p = 1.54). Figure 2B 
displays the high and low valence states using the Brain 
Vision and Emotiv system. A two-way analysis of 
variance revealed that the observed differences in 
condition (high valence vs. low valence) was not 
statistically significant for either system; differences in 
frequency band (Delta, Theta, Alpha, Beta, Gamma) 
were significant for the Brain Vision system only (F 
(1,31) = 26.690, p < 0.01); and the interaction between 
valence condition and frequency was significant for the 
Brain Vision system only (F (1,31) = 40.048, p < 0.01). 

3.3 Alerting correlations of Epoc and BV 

Correlational data (Table 1 & 2) below provided some 
alerting trends among the different systems and tasks. 
Here we used the approach outlined by Rosnow, 
Rosenthal, and Rubin (2000) which describes the use of 
leveraging aggregate correlations of group means 
rather than individual scores for identifying overall 
trends and assessing validity [ 151 . Using this alerting 
correlation method, we identified trends among ERBP 
and the different task conditions. In both Table 1 and 
Table 2 it can be noted that the Epoc ERBP has a very 


poor and often inverse relationship to that of the BV 
which questions the validity of the Epoc. 

4. DISCUSSION & CONCLUSIONS 

The results of this preliminary study suggest that the 
wireless 14-electrode Epoc system can be used to 
show small differences in general electrical activity 
between psychophysiological states of high and low 
arousal, and high and low valence (particularly in the 
delta, theta, and alpha frequency ranges). The 
observed differences, however, are not statistically 
significant such as with the stationary Brain Vision 
device. Therefore, the use of the Epoc device lacks the 
degree of fidelity that is required to properly 
distinguish these states within a personalized design 
which would be very undesirable in a research setting. 
The findings offer evidence against the validity of 
consumer-grade systems for research grade tasks, 
although their low fidelity ability to distinguish states 
may still be useful for some EEG-based BCI 
applications. The Epoc, and similar consumer-grade 
devices with a limited number of electrodes could be 
strongly improved simply by designing a product which 
adheres to conventional EEG electrode placement. 
Using the common electrode placements would lead to 
improved comparative capabilities with traditional EEG 
devices/existing EEG studies, better-referencing 
montages, and increased translatability of anatomically 
related conclusions. 

During the ANEW and IAPS tasks whereas the Brain 
Vision system detected high levels of gamma 
frequency band activity, the Epoc system consistently 
failed to replicate this finding. We hypothesize that 
this difference can be attributed to technical aspects of 
the 14-electrode setup such as the lack of a signal 
amplifier. The Brain Vision system uses an ActiChamp™ 
signal amplifier which enables it to detect higher 
frequencies of electrophysiological activity—higher 
beta oscillations and gamma oscillations. In contrast, 
the Epoc wireless headset transmits 
electrophysiological recordings directly from the scalp 
to a USB-A dongle in the signal processing computer. 
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Table 1. Alerting Pearson Correlations of ERBP for each condition and EEG system (Brain Vision, BV, vs. Epoc 
Emotiv) using the IAPS Task. 



EEG 

Frequency 

Epoc Low 
Arousal 

Epoc High 
Arousal 

BV Low 

Arousal 

BV High 
Arousal 

Epoc Low Epoc High 
Valence Valence 

BV Low 

Valence 

BV High 
Valence 

EEG 

1 









Frequency 










Epoc Low 

-.887* 

1 








Arousal 

(0.045) 









Epoc High 

-.930* 

.946* 

1 







Arousal 

(0.022) 

(0.015) 








BV Low 

0.658 

-0.468 

-0.586 

1 






Arousal 

(0.227) 

(0.427) 

(0.299) 







BV High 

0.666 

-0.489 

-0.624 

.995** 

1 





Arousal 

(0.219) 

(0.403) 

(0.260) 

(0.000) 






Epoc Low 

-.881* 

992** 

.908* 

-0.412 

-0.423 

1 




Valence 

(0.048) 

(0.001) 

(0.033) 

(0.491) 

(0.478) 





Epoc High 

-.923* 

.989** 

.983** 

-0.548 

-0.575 

.967** 

1 



Valence 

(0.025) 

(0.001) 

(0.003) 

(0.339) 

(0.310) 

(0.007) 




BV Low 

0.553 

-0.297 

-0.471 

.975** 

g74** 

-0.231 

-0.400 

1 


Valence 

(0.334) 

(0.627) 

(0.423) 

(0.005) 

(0.005) 

(0.708) 

(0.505) 



BV High 

0.523 

-0.252 

-0.449 

.954* 

.958* 

-0.180 

-0.364 

.996** 

1 

Valence 

(0.366) 

(0.683) 

(0.448) 

(0.012) 

(0.010) 

(0.772) 

(0.548) 

(0.000) 



*. Correlation is significant at the 0.05 level (2-tailed). 
**. Correlation is significant at the 0.01 level (2-tailed). 
1 Exact P-Values for correlations are in parentheses. 


This interface for amplifying signals may be in part 
responsible for making higher beta and gamma 
oscillations harder to detect. A more sophisticated 
amplification would drastically improve the signal 
quality of consumer devices, but the cost required to 
do so would not be consumer friendly. 

For the ANEW task, differences in frequency band 
amplitude between high and low valence states were 
not significant, whereas the interaction between 
valence condition and frequency was significant. This 
discrepancy is likely due to the close proximity of the 
emotion-evoking ratings in the high valence and low 
valence conditions. Future studies should examine this 
result using a greater difference between the mean 
rating of the words used in the two conditions of the 
ANEW task. 


In conclusion, we demonstrate further evidence for a 
consumer EEGs' inability to reproduce differences that 
are distinctly obtainable using a research EEG device 
[161 . We were able to demonstrate this with a small 
number of subjects by using the statistical technique of 
Vincentization. Given that the goal of most emerging 
approaches using EEG-based machine learning and 
categorization algorithms is ultimately intended for 
personalized individual consumer use, the fact that the 
present perspective focused on a demonstration based 
on a small sample size cannot be considered as a major 
limitation. The experiment was done at the same grade 
(scale) as that of the intended consumer usage: in 
other words, it focusses on usability in one person, not 
groups of people. 
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Overall, we urge caution to researchers planning to use 
a consumer device for important research experiments 
but do acknowledge that devices like the Emotiv have 
some benefits such as being wireless, quick to set up, 
are extremely portable, and could still be useful in 
some BCI applications where signal quality and validity 


are not paramount. For instance, exerting BCI control 
over an external device (i.e. wheelchair, prosthetic 
arm) can be done by calibrating any signal to a function 
of the device (i.e. move forward, backward, side to 
side, extend/retract reach). 


Table 2. Alerting Pearson Correlations of ERBP for each condition and EEG system (Brain Vision, BV, vs. Epoc 
Emotiv) using the ANEW task. 



EEG 

Frequency 

Epoc Low 
Arousal 

Epoc High 
Arousal 

BV Low 

Arousal 

BV High 
Arousal 

Epoc Low Epoc High 
Valence Valence 

BV Low 

Valence 

BV High 
Valence 

EEG 

1 









Frequency 










Epoc Low 

-0.799 

1 








Arousal 

(0.105) 









Epoc High 

-0.809 

.992** 

1 







Arousal 

(0.098) 

(0.001) 








BV Low 

0.444 

0.099 

0.134 

1 






Arousal 

(0.454) 

(0.874) 

(0.830) 







BV High 

0.682 

-0.241 

-0.202 

.932* 

1 





Arousal 

(0.205) 

(0.696) 

(0.745) 

(0.021) 






Epoc Low 

-0.811 

0.790 

0.856 

0.040 

-0.234 

1 




Valence 

(0.096) 

(0.112) 

(0.064) 

(0.949) 

(0.704) 





Epoc High 

-.912* 

.928* 

.958* 

-0.037 

-0.329 

.926* 

1 



Valence 

(0.031) 

(0.023) 

(0.010) 

(0.953) 

(0.589) 

(0.024) 




BV Low 

0.424 

0.124 

0.159 

.992** 

.901* 

0.083 

-0.020 

1 


Valence 

(0.477) 

(0.842) 

(0.798) 

(0.001) 

(0.037) 

(0.894) 

(0.975) 



BV High 

0.431 

0.124 

0.152 

.997** 

.928* 

0.021 

-0.025 

.982** 

1 

Valence 

(0.469) 

(0.843) 

(0.807) 

(0.000) 

(0.023) 

(0.973) 

(0.968) 

(0.003) 



*. Correlation is significant at the 0.05 level (2-tailed). 
**. Correlation is significant at the 0.01 level (2-tailed). 
1 Exact P-Values for correlations are in parentheses. 


Notice, in this example the ability for the BCI to 
differentiate valence/arousal emotional states is not 
necessary. In this case, the goal is to make the system 
successfully work for the user, not to produce valid 
signatures of emotion. On the other hand, if the goal of 
a study has anything to do with emotional states, like 
using an Epoc to investigate clinical neuropsychiatric 
populations for emotional dysregulation in mood 
disorders or any cognitive neuroscience experiment of 
emotion in which you need to be confident in the 
validity of your signal, the consumer-grade devices 


must be cautioned against. Therefore, we argue that 
the issue is not so much that consumer devices have 
no place in the research laboratory, but rather that 
their use and data interpretation need to be done 
appropriately in contexts in which they are valid. Upon 
this investigation, using an Epoc EEG device to 
determine emotional states in individuals would have 
questionable validity as it is not reproducible using 
research grade EEG. The validity should not be 
assumed as simply coming in a box, together with the 
device! 
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Supplementary Materials: Additional information regarding 
the EEG signal processing are available online at 
https://doi.org/10.31117/neuroscirn.v2nl.21. 
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